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Abstract 

We show that a family of random variables is uniformly integrable if and 
only if it is stochastically bounded in the increasing convex order by an in- 
tegrable random variable. This result is complemented by proving analogous 
statements for the strong stochastic order and for power-integrable domi- 
nating random variables. Especially, we show that whenever a family of 
random variables is stochastically bounded by a p-integrable random vari- 
able for some p > 1, there is no distinction between the strong order and 
the increasing convex order. These results also yield new characterizations of 
relative compactness in Wasserstein and Prohorov metrics. 
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1 Introduction 

Let be a sequence of random variables such that X n — > X almost surely. 

Lebesgue's classical dominated convergence theorem implies that 

E\Xr,-X\^Q (I) 



if there exists an integrable random variable Y such that 

\X n \ < Y almost surely for all n. 
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A probabilistic version (e.g. [17, Thm. 9.1]) of the above condition is to require 
that 



where < st denotes the strong (a.k.a. usual) stochastic order [11, 14, 16]. It is 
well known that in general (2) is not necessary for (1). We will show that a 
sharp characterization can be obtained when the strong stochastic order in (2) is 
replaced by a weaker one. Namely, we will show that (1) holds if and only if there 
exists an integrable random variable Y such that 



where <i cx denotes the increasing convex stochastic order [14, 16]. 

More generally, our first main result, Theorem 1, shows that a family of random 
variables {^ n } is uniformly integrable if and only if (3) holds. For almost surely 
convergent random sequences, this result yields the equivalence of (1) and (3) 
(e.g. [8, Prop 4.12]). From the analysis point of view, the characterization of 
uniform integrability in terms of the increasing convex order can be seen as a 
new way to represent domination in Lebesgue's dominated convergence theorem 
in the natural manner. What makes this result important in probability is that 
convex and increasing convex orders are intimately connected with the existence 
of martingales and submartingales with given marginals (e.g. Kellerer [9]; Hirsch 
and Yor [7] and references therein). 

The second main result, Theorem 2, shows that when studying whether a 
family of random variables is stochastically bounded by a random variable in LP 
for some p > 1, there is no need to distinguish between the strong order and the 
increasing convex order. This somewhat surprising result, which is a consequence 
of a Hardy-Littlewood maximal inequality, may open new ways to establishing 
strong stochastic bounds using tools of convex analysis. 

The main results are complemented by Proposition 1, which states that {^ n } 
is tight if and only if (2) holds for some (almost surely finite) random variable 
Y. This simple result is probably well known, because 'bounded in probability' 
and 'stochastically bounded' are commonly used as synonyms for 'tight'. We have 
formulated it here explicitly in order to complete the big picture on various stochas- 
tic boundedness relationships in Theorem 3. The implication diagram in Figure 1 
summarizes the findings and illustrates how these concepts are related to relative 
compactness with respect to Wasserstein and Prohorov metrics. Figure 1 provides 
a new unified view to earlier studies on lattice properties of increasing convex and 
strong stochastic orders (Kertz and Rosier [10]; Miiller and Scarsini [13]). 

The rest of this paper is organized as follows. Section 2 introduces definitions 
and notation. Section 3 discusses uniform integrability and tightness of positive 
random variables, and Section 4 extends the analysis to power-integrable ran- 
dom variables. Section 5 summarizes the main results in a diagram (Figure 1) 
and presents counterexamples confirming the sharpness of the implications in the 
diagram. 



X n \ <st Y for all re, 




X n \ <icx Y for all n 



(3) 



2 



2 Definitions and notation 



2.1 Uniform integrability and tightness 

In general, we shall assume that {Aq,} is a family of random variables with values 
in the (i-dimensional Euclidean space indexed by a parameter a taking values 
in an arbitrary set. The random variables do not need to be defined on a common 
probability space; instead, we assume that each X a is a random variable defined 
on some probability space (O a , F a , P a ). To keep the notation light, we replace P a 
by P, and denote the expectation with respect to P a by E. In addition, we denote 
by \x a = P o X^ 1 the distribution of X a defined on the Borel sets of R d . When 
d = 1, the distribution function of X a is defined by F a (t) = P(X a < t), and the 
quantile function by = inf{t : F a (t) > u} for u S (0, 1). 

A family of random variables {A Q } is called: 

• uniformly integrable if sup a £?|X a |l(|X Q | > t) — > as t — > oo, 

• uniformly p-integrable if sup Q £^|X a | p l(|Jr a | > t) — > as t — > oo, 

• tight if sup Q P(|X Q | > t) — >• as t — > oo, and 

• bounded in LP if sup Q ^|X a | p < oo. 

Remark 1. All results in this article remain valid also in a slightly more general 
setting where R rf is replaced by a complete separable metric space S satisfying the 
Heine-Borel property: a subset of S is compact if and only if it is bounded and 
closed. In this case we replace the norm of x by \x\ = p(x,xo), where p is the 
metric and xq is an arbitrary reference point in S. For concreteness, we prefer to 
formulate the results for M rf -valued random variables. 

2.2 Stochastic orders 

For real- valued random variables X\ and X2, we denote X\ < s t X2 and say that 
X\ is less than X2 in the strong stochastic order, or X% is st-bounded by X%, if 
Ecj)(Xi) < E(j){X2) for all increasing 1 measurable functions (j) : R — > M+. It is well 
known [11, 14, 16] that the following are equivalent for any real- valued random 
variables X\ and X2 with distribution functions F\ and F2: 

(i) X l < st X 2 . 

(ii) 1 - Fx(t) < 1 - F 2 (t) for all t € M. 

(iii) Ff x (n) < F 2 -1 (u) for all « G (0, 1). 

x The symbol R+ refers to positive real numbers including zero. In general, we use the terms 
'positive', 'increasing', and 'less than' as synonyms for 'nonnegative', 'nondecreasing', and 'less 
or equal than'. 
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(iv) There exist random variables X\ and X2 defined on a common probability 
space, distributed according to X% and X2, such that X% < X2 almost surely. 

For positive random variables X% and X2, we denote X\ <i cx X2 and say that 
Xi is less than X2 in the increasing convex order, or X\ is icx-bounded by X2, if 
E(f)(X) < E(f>(Y) for all increasing convex functions (j> : R4. — > R+. A fundamental 
characterization (e.g. [14, Thm. 1.5.7]) of increasing convex stochastic orders 
states that 



Jt 

denotes the integrated survival function of Xi, and x + = max(x, 0). Basic technical 
facts about integral survival functions are given in Appendix B. 

Remark 2. Note that if EX2 = 00, then any positive random variable is icx- 
bounded by X2. Therefore to say that a random variable X\ is icx-bounded 
by X2 truly makes sense only when X2 is integrable, in which case also X\ is 
integrable. 

Remark 3. The notion of increasing convex order can also be extended to random 
variables on the full real line having an integrable right tail. Because the main 
results of this article concern norms of random variables, for simplicity we prefer 
to restrict our terminology to positive random variables, using 'integrable' in place 
of 'having an integrable right tail' etc. 

3 Uniform integrability and tightness of positive ran- 
dom variables 

The following theorem, the first main result of this article, provides a sharp char- 
acterization of uniform integrability as stochastic boundedness in the increasing 
convex order. We say that a family of positive random variables is icx- 

bounded by a random variable Y if X a <i cx Y for all a. 

Theorem 1. The following are equivalent for any family of positive random vari- 
ables {X a } with distribution functions F a : 

(i) {X a } is icx-bounded by an integrable random variable. 

(ii) {X a } is uniformly integrable. 

(Hi) lim^oo sup Q / t °°(l - F a (u)) du = 0. 



X\ <i cx X% 



if and only if H x (t) < H 2 (t) for all t > 0, 



where 
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Proof, (i) ==> (ii). Assume that X a <i cx X for all a, where X has a finite mean. 
Then E(X a - t)+ < E(X - t) + for all t > 0. Note that 

xl(x >t)< 2(x - t/2)+ 

for all x and t. Therefore, 

EX a l(X a >t)< 2E(X a - t/2)+, 

which implies that 

supEX a l(X a >t)< 2E(X - t/2)+. 

a 

Because X has a finite mean, we see by using dominated convergence that the 
right side above tends to zero as t grows, and {A a } is hence uniformly integrable. 

(ii) ==>• (iii). Because 

/oo 
(1 - F a (u)) du = E(X a - t)+ < EX a l(X a > t) 

for all t > and all a, the uniform integrability of {A^} implies (iii). 

(iii) =>■ (i). Define H(t) = sup Q H a (t), where H a (t) = E(X a — t)+ is the 
integrated survival function of X a . Note that H a is convex and H a (t) +t > H a (0) 
for all i > by Lemma 3 in Appendix B. As a consequence, the same properties 
are valid for H. Further, H(t) — » as t — > oo by (iii). Therefore, by Lemma 3 
we conclude that H is the integrated survival function of an integrable positive 
random variable X. By the definition of H, it follows that for all a, 

E(X a - t)+ = H a (t) < H(t) = E{X - t)+ for all t > 0, 

which is equivalent to X a <; cx X [ , Thm. 1.5.7]. □ 

Remark 4. The role of increasing convex functions in characterizing uniform 
integrability is also visible in the de la Vallee-Poussin theorem, which states that 
a family of random variables {A^ Q } in W 1 is uniformly integrable if and only if 

sup E(f>(\X a \) < oo 

a 

for some increasing convex function (j) such that (j)(t)/t — > oo as t — > oo [4, 
Prop. A.2.2]. 

The following simple result characterizes tightness as stochastic boundedness 
in the strong order. We say that a family of random variables {A^} is st-bounded 
by a random variable Y if X a < st Y for all a. A positive random variable X (which 
by convention is allowed to take on the value oo) is called finite if P{X < oo) = 1. 

Proposition 1. The following are equivalent for any family of positive random 
variables {X a } with distribution functions F a : 
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(i) {X a } is st-bounded by a finite random variable. 

(ii) {X a } is tight. 

(Hi) limbec sup a (l - F a (t)) = 0. 

Proof, (i) ==> (ii). Assume that X a < st X for all a, where X is a finite random 
variable. Then 

supP(X a >t)< P(X > t). 

a 

Because the right side above tends to zero as t — > oo, tightness follows, 
(ii) =>■ (hi). Clear. 

(hi) ==> (i). Let F(t) = mi a F a {t). Then Lemma 2 in Appendix A shows 
that F is a distribution function of a finite positive random variable X*. Because 
1 - F a (t) < 1 - F(t) for all t and all a, it follows that X a < st X* for all a. □ 

Remark 5. An alternative equivalent characterization of tightness, analogous to 
Remark 4, is the following: A family of random variables {A^} in W d is tight if 
and only if 

sup Ecj)(\X a \) < oo 

a 

for some positive function <p such that 4>{t) — > oo as t — > oo [12, Lem. D.5.3]. 

4 Stochastic boundedness of power-integrable random 
variables 

4.1 Boundedness in strong vs. increasing convex order 

If a family of positive random variables is st-bounded by a p-integrable random 
variable for some p > 0, then the same is obviously true when 'st-bounded' is 
replaced by 'icx-bounded'. The following result shows that, rather surprisingly, 
these two properties are equivalent when p > 1. 

Theorem 2. For any p > 1, a family of positive random variables {X a } is icx- 
bounded by a p-integrable random variable if and only if {X a } is st-bounded by a 
p-integrable random variable. 

To prove this result, we will apply Hardy-Littlewood maximal functions. Let 
X be an integrable random variable with distribution function F, and denote the 
quantile function of X by F~ l (u) = inf{t : F(t) > u} for u G (0, 1). Let U be a 
uniformly distributed random variable in (0, 1) and define 

X* = MF^(U), 

where 

MF~ 1 (u)= sup (w-u)- 1 F- 1 (v)dv = (l-u)~ 1 F -1 (u)du 

u<w<l Ju Ju 
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is the Hardy-Littlewood maximal function of i 7-1 [6]. The function MF~ l is well- 
defined when X is integrable [3, 10]. We call X* the Hardy-Littlewood maximal 
random variable associated with X. A remarkable result of Kertz and Rosier [10, 
Lem. 4.1] states that for any integrable random variables X and Y, 

X < icx Y if and only if X* < st Y*. (4) 

Lemma 1. X < st X* for any integrable random variable X . 

Proof. Observe that the quantile function of X* equals MF~ l . Further, the mono- 
tonicity of F^ 1 shows that -F -1 (-u) < MF~ l {u) for all u € (0,1). This pointwise 
ordering of the quantile functions is a well-known necessary and sufficient con- 
dition for the strong ordering of the corresponding random variables (e.g. [14, 
Thm. 1.2.4]). □ 

Proof of Theorem 2. Assume that X a <i cx Y for all a, where Y € L p . Then 
<st Y* for all a due to (4), so by applying Lemma 1 we see that the family 
is st-bounded by Y*. It remains to verify that Y* is in LP . Note that the 
quantile function of Y* equals MFy 1 , where Fy 1 is the quantile function of Y. A 
classic inequality of Hardy and Littlewood [G] now implies that 

E(Y*Y = f {MFy l {u)) p du<c f {Fy 1 ^ du = cEY p , 
Jo Jo 

where c = (^rr) • D 



4.2 Power-integrable strong bounds 

The following result shows how to reduce the study of strong boundedness by 
a p-integrable random variable to strong boundedness by an integrable random 
variable. A corresponding statement is not true for the increasing convex ordering, 
see Theorem 3 in Section 5. 

Proposition 2. For any p > and any family of positive random variables {X a } 
with distribution functions F a , the following are equivalent: 

(i) {X a } is st-bounded by a p -integrable random variable. 

(ii) {Xa} is st-bounded by an integrable random variable. 
(Hi) J °° sup Q (l - F a (t^P)) dt < oo. 

Proof, (i) ==> (ii). Assume that X a < s t Y for all a, for some p- integrable random 
variable Y. Because the random variables X a are positive, it follows that Y > 
almost surely. Because x i— > x p is increasing on M + , it follows that X& < s t Y p f° r 
all a. 

(ii) (i). The same argument as above works also in this direction, because 
x i — y x l l p is increasing. 



7 



(i) ==>• (iii). Assume that X a < s t Y for all a, for some p-integrable ran- 
dom variable Y > 0. Define F(t) = inf a F a (t). Proposition 1 implies that 
lim^oo F(t) = 1, so by Lemma 2 in Appendix A, we see that F is a distribu- 
tion function of a finite positive random variable Z. Because Z < st X a for all 
a, it follows that Z < s t Y. The monotonicity of x h- > x p further implies that 
SZP < FY? < oo. Therefore, 

/"OO /"OO 

/ sup(l-F Q (t 1/p ))dt = / (l-F(t 1 / p ))dt = EZ p < oo. 

Jo a Jo 

(iii) ==> (i). Define F(t) = ini a F a (i), and observe that lim^oo F(t) = 1 
by (iii). Again, by Lemma 2, we see that F is the distribution function of some 
finite positive random variable Z, and X a < st Z for all a. Further, the formula 



EZ V = / (l-F{t x / p ))dt 
Jo 



shows that Z is p-integrable. □ 

We conclude this section by a simple corollary of the above result. 

Proposition 3. Let < p < q. If a family of positive random variables {X a } is 
bounded in L q , then it is st-bounded by a p-integrable random variable. 

Proof. Markov's inequality implies that 

1 - F a (t^ p ) = P{\X a \ q > t q/p ) < t- q l p E\X a \ q 
for all a. Therefore, 

sup(l - Fait 1 /?)) < max(l, ct~^ p ), 

a 

where c = sup Q E\X a \ q . Because the right side above is integrable, the claim 
follows by Proposition 2. □ 



5 Summary 

This section concludes the paper with a diagram (Figure 1) which summarizes the 
relationships between the various stochastic boundedness properties. To illustrate 
how these concepts relate to compactness properties of probability measures, let 
us recall the definitions of Prohorov and Wasserstein metrics. 

The Prohorov metric on the space M of probability measures on is defined 

by 

d P (fj,, v) = inf {e > : n(B) < v(B e ) + e and v{B) < fi(B e ) + e for all B G B} , 

where B € = {x € M. d : \x — b\ < e for some b E B} denotes the e-neighborhood of 
B, and B denotes the Borel sets of M. d . The space (M, dp) is a complete separable 
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metric space, and convergence with respect to dp corresponds to convergence in 
distribution (e.g. [ , Thm. 3.1.7]). 

For p > 1, denote by M p the space of probability measures on M. d with a finite 
p-th moment. The p-Wasserstein metric on M p is defined by 

dw, P (l J '' u ) = ( inf / \x-y\ p l{dx,dy)\ 

where K(fi, v) is the set probability measures on W 1 x M. d with first marginal \i and 
second marginal v. The space (M p , dw.p) is a complete separable metric space, and 
a sequence converges with respect to dw,p if an d only if it is uniformly p-integrable 
and converges in distribution [1, Prop. 7.1.5]. See Rachev and Riischendorf [15] 
for a detailed account of dual characterizations of dw, P , and Gibbs and Su [5] for 
a nice survey of different probability metrics and their relationships. 

Theorem 3. For any 0<p<l<g<oo and for any family of random vari- 
ables {X a ~\ in M d with probability distributions fi a = P o X~ l , the implications in 
Figure 1 are valid. In general, no other implications hold. 

Proof. Let us start by verifying the equivalences on the left of Figure 1, starting 
from top. The first and fourth and due to Proposition 2. The second, third, and 
fifth are due to Theorem 1. The sixth is due to Proposition 1. 

Let us next verify the equivalences on the right, again starting from the top. 
The first is due to Theorem 2. By applying Ambrosio, Gigli, and Savare [1, 
Prop. 7.1.5], we know that {u Q } is relatively compact in the g-Wasserstein metric 
if and only if {X a } is uniformly g-integrable and tight. Because closed bounded 
sets are compact in M. d , it follows that uniform q-integrability implies tightness. 
Therefore, the second equivalence on the right holds. The third equivalence fol- 
lows by observing that the L^-norm of X a is equal to the g-Wasserstein distance 
between fj, a and the Dirac measure at zero. The fourth and fifth equivalences are 
verified by similar reasoning. For the sixth, it suffices to recall that tightness and 
relative compactness in the Prohorov metric are equivalent by Prohorov's classic 
theorem (e.g. [8, Thm 16.3]). 

The downward implications are more or less immediate. Starting from top, 
the first, fourth, and seventh are trivial, because strong ordering implies increas- 
ing convex ordering. The second, fifth, and eighth are trivial as well, because 
increasing convex ordering implies the ordering of the means. The third, sixth, 
and ninth are due to Proposition 3. 

To complete the picture, we will now construct families of random variables 
which show that none of the one-way implications in Figure 1 can be reversed in 
general. Let U be a uniformly distributed random variable in (0, 1), and let (j) n 
and ip n , n > 2, be random variables such that 

[ n with probability In with probability (nlogn) -1 , 

I else, I else. 
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{|JT Q | 9 } is st-bounded 
by an integrable r.v. 

{X a } is uniformly 
g-integrable 



{X a } is uniformly 
integrable 



is st-bounded 
by an integrable r.v. 

is uniformly 
p-integrable 



{X a } is tight 



{\X a \} is st-bounded by 
a g-integrable r.v. 

{|X Q | 9 } is icx-boundcd 
by an integrable r.v. 

{X a } is bounded 
in L" 

{|X Q |} is st-bounded by 
an integrable r.v. 

is icx-boundcd 
by an integrable r.v. 

{X a } is bounded 
in L 1 

{|X Q |} is st-bounded by 
a p-integrablc r.v. 

{|X Q | P } is icx-boundcd 
by an integrable r.v. 

a- 

{X a } is bounded 
in LP 

\ 

{|X Q |} is st-bounded 
by a finite r.v. 



{|X Q |} is icx-bounded 
by a g-integrable r.v. 

{^ a } is rel. compact in 
the g-Wasserstein metric 

{Ha} is bounded in 
the g-Wasserstein metric 



{/z a } is rel. compact in 
the 1-Wasserstcin metric 



{^a} is bounded in 
the 1-Wasserstcin metric 



{fj, a } is rel. compact in 
the Prohorov metric 



Figure 1: Stochastic boundedness relationships. 



Then for any < p < 1 < q: 

• {e 1 / 17 } is st-bounded by a finite r.v. but not bounded in LP. 

• {0r/ P } is bounded in LP but not uniformly p-integrable. 

• {ifn P } is uniformly p- integrable but not st-bounded by a r.v. in LP . 

• {U^ 1 } is st-bounded by a r.v. in L p but not bounded in L 1 . 
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• {(j) n } is bounded in L 1 but not uniformly integrable. 

• {V'n} is uniformly integrable but not st-bounded by an integrable r.v. 

• {U^ 1 ^} is st-bounded by an integrable r.v. but not bounded in L q . 

• {0r/ 9 } is bounded in L q but not uniformly q- integrable. 

• {iftn } is uniformly g-integrable but not st-bounded by a r.v. in L q . 

The straightforward computations required for verifying the above claims are left 
to the reader. □ 

A Distribution functions 

If X is a finite positive random variable, then its distribution function F(t) = 
P(X < t) is increasing, right-continuous, and satisfies F(0) > and lim^oo F(t) = 
1. Recall that any function F with these properties can be realized as the dis- 
tribution function of the random variable X = F~ l (U), where F (u) = inf{i £ 
R+ : F(t) > u} and U is a uniformly distributed random variable in (0, 1). 

Lemma 2. For any family {F a } of distribution functions on R +; the function 
F(t) = inf a F a (t) is a distribution function on M + if and only if 

lim inf F a (t) = 1. (5) 

t— >oo a 

Proof. The necessity of (5) is obvious. To prove sufficiency, assume that a family 
of distribution functions {F a } on M + satisfies (5). Then it immediately follows 
that F is increasing, F(0) > 0, and lim^oo F(t) = 1. Therefore, we only need to 
verify that F is right-continuous. Fix arbitrary t > and e > 0, and choose an 
index a such that F a (t) < F(t) + e/2. By the right-continuity of F a there exists 
a 5 > such that F a (t + h) — F a (t) < e/2 for all h G (0, 5). As a consequence, 

F(t + h)< F a (t + h)< F Q (t) + e/2 < F(t) + e 

for all h € (0,(5). Because F is increasing, this implies that F is right-continuous. 

□ 

B Integrated survival functions 

The integrated survival function of a positive integrable random variable X with 
distribution function F is defined by 

/oo 
(1-F(u))du. 
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The following result characterizes the family of integrated survival functions gener- 
ated by positive integrable random variables. The proof is similar to an analogous 
characterization for random variables on the full real line (Miiller and Stoyan [14, 
Thm. 1.5.10]). Because our formulation of the result is slightly different, we in- 
clude the proof here for the reader's convenience. 

Lemma 3. A function H : R + — > R + is the integrated survival function of a 
positive integrable random variable if and only if 

(i) H is convex, 

(ii) Y\-m t ->ooH(t) = 0, 

(Hi) H(t) + 1 > H(0) for allt>0. 

Proof. Assume first that H is the integrated survival function of a positive inte- 
grable random variable X. Then (i) follows immediately because t H > (x — t) + is 
convex for all x, and (ii) follows by dominated convergence. To see the validity 
of (iii), it suffices to observe that 

H{t) + t = Emax(X,t) > EX = H(0). 

Assume next that a function H : R + —> R + satisfies (i)-(iii). The convexity of 
H together with (ii) implies that H is decreasing on R + . By applying (iii) we see 
that —t < H(t) — H(0) < for all t, which implies that H is continuous at zero. 
As a consequence, the right derivative H',(t) of H exists for all t > 0, and the 
function H' + : M + — > R is right-continuous and increasing (e.g. [2, Prop. 4.1.1]). 
Define F(t) = 1 + H' + (t). Then F(0) > due to (iii), and lim^ 00 F(t) = 1, 
because H(t) decreases to zero as t — > oo. Because F is right-continuous and 
increasing, we conclude that F is the distribution function of the random variable 
X = F _1 ([/), where U is uniformly distributed in (0,1). To see that H is the 
integrated survival function of X, we note by changing the order of integration 
that for all t, 

poo poo 

H(t) = - H' + (u)du= (1 -F{u))du = E(X -t)+. 
Jt Jt 

Especially, EX = H(0) is finite, which shows that X is integrable. □ 
Lemma 4. For any integrable positive random variable X and for any p > 1, 

poo 

EX p =p / H{t l /( p -^)dt, 
Jo 

where H(t) =E{X -t) + . 
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Proof. Denote G{u) = P(X > u). Because P(X P > u) = G^ 1 ^), a change of 
variables shows that 

/*00 /*00 

EX P = / G(«Vp) du = p G(u)u e du, 
Jo Jo 

where e = p — 1. Further, by writing u e = J °° l(t 1//<: < u) dt and changing the 
order of integration, we find that 

pOO /*00 / poo \ poo 

pj G(u)u e du = pJ I J G{u)du\dt=pJ H(t 1/e )dt. 

□ 
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